Charged particle beam apparatus that irradiates charged particles as probes to a specimen and obtains specimen information about an irradiated position by detecting charged particles generated from the irradiated position have been widespread. Well known as a very commonly used charged particle apparatus is a scanning electron microscope (SEM), in which two-dimensional scanning is performed on a specimen using an electron beam probe converged onto the specimen, and then by detecting secondary electrons and backscattered electrons generated from irradiated positions, and by mapping signals about the amount of detected electrons in synchronization with the scanning of the probe, a two-dimensional image of the scanning region can be obtained. Plural techniques concerning a scanning electron microscope in which signal electrons are detected through energy discrimination (classification) in order to emphasize a desired contrast such as a concave-convex contrast or a composition contrast are disclosed in the following five patent literatures.